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ABSTRACT 

Thi s paper addres ses difficult issues in evaluat i on 
of college faculty for scholarship, teaching, and service 
particularly noting areas that trouble faculty themselves. A section 
or. evaluation of teaching notes long standing debate and 
disagreement. This section then lists many of the central questions 
around evaluation of teaching and notes that some faculty pay little 
attention to teaching in order to focus more on research and 
scholarship. A discussion of evaluation of scholarship considers the 
following issues: the problems of publishing; the difficulty of 
placing articles with reputable journals; the different status and 
quality levels of conferences where papers are presented; the 
difficulty of evaluating the quality of published pieces; the 
possibility of faculty working with presses in a relationship where 
the author virtually pays to have his or her book published; and the 
mining of data from vast research projects for information of perhaps 
questionable usefulness. The section on evaluating faculty service to 
the institution describes the very extensive time demands of advising 
and committee work and the difficulties connected with effectively 
evaluating such work. A final section notes that often the 
administrators evaluating a faculty member f s research productivity do 
not have the expertise to accurately assess scholarship. (JB) 
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ABSTRACT 

THIS PAPER REVIEWS SOME OF THE MOST CRITICAL ISSUES IN 
TERMS OF THE EVALUATION OF COLLEGE FACULTY IN UNIVERSITIES 
ACROSS AMERICA AND THE WORLD. SPECIFIC ATTENTION IS FOCUSED 
ON THE RESEARCH , TEACHING, AND SERVICE DOMAINS . MAJOR 
CONCERNS ARE RAISED REGARDING THE EVALUATION OF TENURED 
FACULTY . 
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THE EVALUATION OF COLLEGE FACULTY MEMBERS CONTINUES TO 
BE A PROBLEMATIC ISSUE YEARLY. ADMINISTRATIVE PERSONNEL AND 
FACULTY SPEND LARGE AMOUNTS OF TIME ENGAGING IN THE 
EVALUATION PROCESS AND DEBATING AND DISCUSSING THE RESULTS 
OF SAID EVALUATION PROCESS. IN ADDITION, THERE IS MOVEMENT 
AFOOT TO EVALUATE TENURED FACULTY ( GOODMAN, 1990 ) . 

THIS PAPER ADDRESSES A NUMBER OF CRITICAL ISSUES 
RELATIVE TO FACULTY EVALUATION. THE SATIRICAL TITLE 
REFLECTS SOME OF THE EXASPERATION THAT FACULTY FEEL 
REGARDING THE EVALUATION OF THEIR SCHOLASTIC WORK I.E. THAT 
THE ADMINI STRATORTS WHO ARE EVALUATING THEM HAVE ABSOLUTELY 
NO IDEA WHAT IT IS THEY ARE RESEARCHING OR WRITING ABOUT AND 
HAVE NO WAY OF CLEARLY, OBJECTIVELY EVALUATING THEM. THIS 
PAPER REVIEWS THE TRIARCHIC CHARGE OF FACULTY MEMBERS AT 
COLLEGES AND UNIVERSITIES- TO ENGAGE IN TEACHING, RESEARCH 
AND SERVICE. THE EVALUATION OF THESE THREE AREAS WILL BE 
COMPREHENSIVELY DISCUSSED WITH PITFALLS IDENTIFIED AND 
CONCERNS ADDRESSED. 

CONCERNS RELATIVE TO TEACHING 

THE EVALUATION OF TEACHERS AND TEACHING HAS A LONG 
NOTORIOUS HISTORY OF DEBATE. DISCUSSION, AND ARGUMENTATION. 
WHAT EXACTLY IS GOOD TEACHING ? HOW DO WE EVALUATE 
COMPETENT TEA*, ^RS? DO STUDENTS HAVE THE NECESSARY SKILLS TO 
CRITICALLY EVALUATE DOCTORAL LEVEL PERSONNEL ? 



SHOULD STUDENTS BE DOING THE EVALUATION OR SHOULD 
PROFESSIONAL STAFF BE HIRED TO DO THIS TASK OR CHORE ? 
WHAT ABOUT THE EVALUATION INSTRUMENT ? SHOULD A FIVE POINT 
LIKERT SCALE BE USED ? CAN ENGLISH BE EVALUATED IN THE SAME 
MODE AS PHYSICAL EDUCATION OR CHEMISTRY ? WHAT ABOUT 
INSTRUCTIONAL TIME THAT IS LOST DUE TO EVALUATIVE EFFORTS ? 
SHOULD ALL CLASSES BE EVALUATED ? SHOULD GRADUATE CLASSES 
AND UNDERGRADUATE CLASHES BE EVALUATED ACCORDING TO THE SAME 
PRINCIPLES OR BY THE f,AME PERSONNEL ? WHAT ABOUT ONE 
FACULTY MEMBER WHO TEACHES ALL FRESHMEN INTRO CLASSES AND 
ANOTHER THAT TEACHES SIMPLY GRADUATE LEVEL CLASSES ? 
THE ISSUES REGARDING THE EVALUATION OF FACULTY MEMBERS AND 
THE PERILS AND PITFALLS OF EVALUATION HAVE BEEN REVIEWED BY 
NEELY AND SHAUGHNESSY (1987) 

BASICALLY, FRESHMEN IN COLLEGE MAY NOT HAVE HAD A GREAT 
MANY » GOOD » TEACHERS DURING HIGH SCHOOL. EVEN AS THEY 
PASS THROUGH COLLEGE, THEY MAY HAVE ONLY MINIMAL CONTACT 
WITH TRULY GREAT TEACHERS WHO ARE ABLE TO MOTIVATE AND 
INSTILL AN INTEREST IN LEARNING. MANY FACULTY MEMBERS GIVE 
TEACHING A LOW PRIOTITY SO THAT THEY CAN PURSUE THEIR OWN 
RESEARCH AND PUBLISING CONCERNS. JOHN KENNETH GALBNRAITH 
HAS WRITTEN AN EXCELLENT BOOK ON THIS TOPIC ENTITLED " A 
TENURED PROFESSOR " ( 1990 ) 

A FACULTY MEMBER IN PHYSICAL EDUCATION MAY PRESENT A 
PAPER AT A NATIONAL CONFERENCE WITH THE TITLE USED IN THIS 



PAPER . MOST ADMINISTRATORS WOULD PRESUME THAT THE PAPER HAD 
SOMETHING TO DO WITH EITHER FOOTBALL OR BASKETBALL OR SPORTS 
OR BOTH, OR SOMETHING TO THAT EFFECT. IN EITHER REGARD, 
THE FACULTY MEMBER HAS ,! ANOTHER THREE LINES " ON HIS OR HER 
RESUME . 

IT IS NOT KNOWN HOW WELL THE PAPER WAS RECEIVED. IT IS 
NOT KNOWN WHETHER 5 OR 50 OR 500 PEOPLE ATTENDED THE 
LECTURE . 

IT MAY HAVE COST THE PROFESSOR A REGISTRATION FEE OF $ 25.00 
OR $ 125.00 TO ATTEND THIS CONFERENCE. NO ONE HAS STUDIED 
( AT LEAST TO MY KNOWLEDGE ) THE CORRELATION BETWEEN STATUS 
OF CONFERENCE AND REGISTRATION FEE. SOME CONFERENCES CHARGE 
A VERY HIGH REGISTRATION FEE AND PUBLISH EITHER THE 
ABSTRACTS OF PAPERS, OR IN SOME INSTANCES THE ENTIRE 
PROCEEDINGS OF THE CONFERENCE. SOME CONFERENCES CHARGE A 
LOWER FEE f BUT THERE IS A DISTINCT REFEREE PROCESS- NOT ALL 
PAPERS ARE ACCEPTED, AND IN FACT, IN SOME CONFERENCES, 5 0 TO 
60 % OF THE PAPERS ARE REJECTED. 

IT IS DIFFICULT TO ASCERTAIN THE QUALITY OF THIS PAPER. 
SOME MAY WAX AND WANE OVER THIS PAPER FOR MANY YEARS AND 
GIVE IT THE ACCOLADES IT DESERVES; FOR OTHERS THIS PAPER 
WILL END UP IN THE BIG GARBAGE CAN IN THE SKY- E.R.I.C.- 
( PLEASE DON'T SUE ME ERIC- ) 

EDUCATIONAL RESOURCES INFORMATION CLEARINGHOUSE OR RESOURCES 
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IN EDUCATION. CLEARLY, MY CALLING IT A » BIG GARBAGE CAN " 
REFLECTS MY OWN VALUES AND STANDARDS- AT THE DOCTORAL OR 
PROFESSORIAL LEVEL. IT MAY CONCEIVABLY BE PUBLISHED 
SOMEWHERE , AT SOME TIME, IN SOME ESOTERIC JOURNAL IN SOME 
FAR FLUNG CORNER OF THE UNIVERSE ( ISSUES IN NORTH DAKOTA 
HIGER EDUCATION ) OR IT MAY NEVER BE PUBLISHED. THERE MAY 
BE 5 REFERENCES AT THE END OF THIS PAPER OR THERE MAY BE NO 
REFERENCES AT THE END OF THIS PAPER. IT IS CONCEIVABLE THAT 
HARVARD EDUCATIONAL REVIEW MAY LIKE THIS PAPER. ( IT MAY NOT 
BE PUBLISHED, BUT THEY MAY LIKE IT ) AND IT COULD ALSO SIT 
ON SOME EDITOR'S DESK FOR TWO YEARS. ( ONE PAPER SUBMITTED 
BY THIS AUTHOR TO A MAJOR REFEREED JOURNAL " SAT " FOR 
APPROXIMATELY FOUR YEARS WHILE TWO EDITORS CHANGED, IN 
ANOTHER INSTANCE, IN THIS AUTHOR'S EXPERIENCE, A PAPER TO BE 
PUBLISHED WAS HELD UNTIL THE JOURNAL WENT OUT OF BUSINESS. 

ONE PAPER PUBLISHED BY THE FIRST AUTHOR RECEIVED GLOWING 
ACCOLADES IN A MAJOR NATIONAL PUBLICATION ( IN SPITE OF THE 
FACT THAT THE SAMPLE SIZE WAS NOTORIOUSLY SMALL ! ) 

WITH GLOBAL INTERNATIONALISM BECOMING A PREVALENT 
CONCERN, MANY FACULTY ARE PUBLISHING IN THE JOURNALS OF 
OTHER COUNTRIES. ONE PAPER THAT WILL BE VERY DIFFICULT FOR 
ADMINISTRATIVE PERSONNEL TO EVALUATE IS " BITI STARS 
SAMOSTOJNO UCECEGA OTROKA " PUBLISHED IN 1991 BY THE FIRST 
AUTHOR AND A GRADUATE STUDENT ( SHAUGHNESSY AND SCOTT , 1991 
) THE PAPER WAS TRANSLATED INTO YUGOSLAVIAN AND PUBLISHED 
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IN A JOURNAL NAMED " EDUCA " .SINCE MOST ADMINISTRATORS AND 
EVEN FACULTY CANNOT READ YUGLO S LAV I AN ( ACTUALLY IT WAS 
PUBLISHED IN SLOVENIAN, A DIALECT OF YUGOSLAVIA ) , IT WILL 
BE DIFFICULT TO DETERMINE THE MERITS OF THE ARTICLE. 
IT DID COMPRISE 5 PAGES OF TYPE, SINGLE SPACE, BUT IT'S 
RELEVANCE HAS YET TO BE DETERMINED. 

A MAJOR CONCERN IS THE CARLTON PRESS OR VANTAGE PRESS 
SYNDROME I.E. PROFESSORS CAN PAY TO HAVE THEIR BOOKS 
PUBLISHED. THERE ARE A NUMBER OF OTHER LESSER KNOWN 
PUBLISHING HOUSES WHICH PUBLISH BOOKS, BUT THEIR NAMES SHALL 
NOT BE MENTIONED HERE TO PROTECT THOSE FACULTY WHO MAY STILL 
NEED THEIR BOOKS PUBLISHED. 

THE » PUBLISH FOR PAY " SYNDROME IS STILL ALIVE AND WELL 
AND STILL WITH US- CALL IT WHAT YOU WANT, THERE ARE STILL 
JOURNALS WHO ASK FOR TYPESETTING FEES, PUBLICATION 
ASSISTANCE FEES, A SUBSCRIPTION TO THEIR JOURNAL OR 
MEMBERSHIP IN THEIR ORGANIZATION AS PREREQUISITES TO 
PUBLICATION. IN SOME INSTANCES, THE MESSAGE IS NOT CLEAR; 
IN OTHER INSTANCES IT IS CLEAR AND DIRECT. 
" SHOTGUN EMPIRICISM " 

FOR THE CONCEPT OF " SHOT GUN EMPIRICISM M I AM 
ENDEBTED TO KIMMO LEHTONEN OF FINLAND WHO PARTICIPATED IN A 
NATIONAL CONFERENCE. A RESEARCHER REPORTED THE RESULTS OF 
HIS 495 QUESTION QUESTIONNAIRE. HE CONTINUED TO EXAMINE 
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EACH QUESTION FROM THE POINT OF VIEW OF VARIOUS VARIABLES - 
I.E., SEX, EDUCATION, SOCIO-ECONOMIC STATUS, AND A HOST OF 
OTHER VARIABLES. HE FINALLY FOUND SOME SIGNIFICANT RESULTS 
AFTER ANALYZING ALL OF THIS DATA. THIS " BUCKSHOT " KIND OF 
APPROACH IS USED BY SOME RESEARCHERS TO ASCERTAIN SOMETHING 
OF SIGNIFICANCE WHEN THEY ARE REALLY NOT SURE WHAT EXACTLY 
IT IS THEY ARE LOOKING FOR- THEY HAVE LOTS AND LOTS OF PAPER 
AND HAVE FOUND SOME SIGNIFICANT STUFF, BUT THE WAY IN WHICH 
THE SIGNIFICANCE WAS PROCURED LEAVES SOMETHING TO BE 
DESIRED. 

" CLASSICAL DUETS " IS ANOTHER CONCEPT TAKEN FROM DR. 
LEHTONEN. THIS SCENARIO ENTAILS TWO PEOPLE- ONE A 
STATISTICIAN AND THE OTHER A RESEARCHER WHO HAS COLLECTED A 
VAST AMOUNT OF DATA. THE STATISTICIAN ANALYZES THE DATA IN 
EVERY CONCEIVABLE MANNER USING EVERY PARA METRIC OR NON 
PARAMETRIC PROCEDURE THAT HE OR SHE CAN THINK OF AND THEN 
THE TWO RESEARCHERS REPORT SIGNIFICANT RESULTS USING 
WHATEVER METHODOLOGY THAT ENJOY MOST. 

LASTLY IS THE " ADAM AND EVE " POSITION TAKEN BY SOME PEOPLE 
WHO FAVOR FACTOR ANALYTIC STUDIES. THEY UTILIZE EVERY 
CONCEIVABLE ROTATION TO FIND SOME SORT OF INTERESTING 
SIGNIFICANT MATERIAL TO WRITE ABOUT. IN ONE STUDY, TWO 
DIAMETRICALLY OPPOSITE FACTORS WERE FOUND TO LOAD ON THE 
SAME FACTOR. IN REVIEWING THIS PIECE THIS AUTHOR BEGAN TO 
FEEL THAT THE RESEARCHER COULD CONCEIVABLY FIND THAT 11 ADAM 



AND EVE » WOULD LOAD ON THE SAME FACTOR. 
THE « SERVICE " COMPONENT 

IN TERMS OF SERVICE, PROFESSORS CONTRIBUTE A GREAT DEAL OF 
TIME TO THE COLLEGE/UNIVERSITY AN J THE COMMUNITY IN TERMS OF 
SERVICE. STUDENT ASSISTANCE AND ADVISING CONSUMES ENORMOUS 
AMOUNTS OF TIME. AFTER CLASS DISCUSSION OFTEN CONSUMES 
GARGANTUAN AMOUNTS OF TIME. FILLING OUT LETTERS OF 
REFERENCE EATS UP VERY LARGE AMOUNTS OF TIME. 

COMMITTEES ARE THE BANE OF THE ACADEMIC COMMUNITY. THEY ARE 
A NECESSARY EVIL, YET AGAIN, HUMONGOUS ( IS THERE SUCH A 
WORD? ) AMOUNTS OF TIME ARE CONSUMED IN INEFFECTIVE, SLOW, 
PONDEROUS, COMMITTEES WHEREIN ISSUES ARE DEBATED AND AGENDAS 
ARE NOT ADHERED TO. SADLY, FOR MANY COLLEGE PROFESSORS, 
THEY HAVE NOT HAD A CLASS IN HOW TO CONDUCT 
EFFICIENT, EFFECTIVE COMMITTEE MEETINGS. 

HOW CAN ADMINISTRATORS EVALUATE ONE'S CONTRIBUTIONS TO THESE 
COMMITTEES ? 

HOW CAN FACULTY DOCUMENT THEIR ATTENDANCE AND CONTRIBUTIONS 
TO THESE MEETINGS ? 

ONE FACULTY MEMBER USED COMMITTEE MEETINGS TO OPEN HIS MAIL 
WHILE ANOTHER GRADED PAPERS DURING THE LONG MONOTONOUS 
REVIEW OF THE MINUTES OF THE LAST MEETING. 

A FACULTY MEMBER, SADLY WAS ON 17 COMMITTEES ONE YEAR. IT 
IS THIS AUTHOR'S BELIEF THAT THIS CONTRIBUTED TO HIS MASSIVE 
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HEART ATTACK. BY DINT OF VARIOUS POSITIONS , SOME FACULTY 
FACE MORE COMMITTEE TIME THAN OTHERS. OTHER FACULTY AVOID 
COMMITTEE DUTY LIKE THE PLAGUE- LEAVING OTHERS TO FACE THESE 
BURDENS . 

THE EVALUATION OF ACADEMIC ADVISING 

ALTHOUGH PROFESSORS ARE GIVEN STUDENTS TO ADVISE, THERE 
IS LITTLE, IF ANY, EVALUATION OF THE ADVISING PROCESS. 
STUDENTS ARE SIMPLY NOT ASKED TO EVALUATE THE QUALITY OR 
QUANTITY OF ADVISING TIME THEY RECEIVE. THERE IS NO 
GENERALLY AGREED UPON INSTRUMENT TO EVALUATE " ACADEMIC 
ADVISING " . YET MUCH TIME IS CONSUMED YEARLY IN THE 
MAJOR/MINOR DEBATE AND THE GENERAL EDUCATION REQUIREMENTS 
AND ELECTIVES CONCERN. IT IS OUR CONTENTION THAT QUALITY 
ADVISING DOES MAKE A DIFFERENCE IN TERMS OF STUDENT SUCCESS. 
SADLY, LITTLE EMPIRICAL DATA IS AVAILABLE TO CORROBORATE 
THIS FACT. 

THE EVALUATION OF RESEARCH 

IN MANY CASES, SOME " EVALUATORS " SIMPLY DO NOT HAVE 
THE TECHNICAL AND STATISTICAL SKILLS TO EVALUATE PUBLISHED 
PAPERS. IN HIS CAPACITY AS EDITOR OF A JOURNAL AND AS 
A REVIEWER FOR SEVERAL INTERNATIONAL JOURNALS, THE FIRST 
AUTHOR HAS BEEN OFTEN FLABBERGASTED IN HIS REVIEW OF PAPERS 
SUBMITTED FOR PUBLICATION. IN MANY INSTANCES, THE ENGLISH 
AND GRAMMAR IS JUST PLAIN POOR AND THERE IS BORING TEPID 
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WRITING. IN OTHER INSTANCES , WE HAVE VERY LONG SENTENCES 
THAT END IN PERCENT - SUCH AS " BLAH BLAH, BLAH BLAH 
PERCENT, BLAH BLAH, BLAH, BLAH, PERCENT " THIS WAS LISTED 
AS AN EMPIRICAL DATA BASED PAPER. IN OTHER INSTANCES 
PAPERS HAVE BEEN RECEIVED THAT CONTAIN DATA THAT SHOULD 
HAVE BEEN ANALYZED IN A NON- PARAMETRIC FASHION, RATHER THAN 
A PARAMETRIC FASHION. IN ONE STUDENT'S MASTER'S THESIS, THE 
STUDENT TRIED TO CONVINCE BOTH HIS MAIN ADVISOR AND THE 
CHAIR OF HIS COMMITTEE THAT IT WAS " OKAY » TO DO " T-TESTS 

WITH THREE INDEPENDENT GROUPS " . FLAGRANT ERRONEOUS 
STATISTICAL ANALYSES ARE OFTEN FOUND, OR MEDIOCRE DATA 
ANALYSES ARE CONDUCTED. 

THERE ARE PROFESSORS WHO CONDUCT COMPLEX STATISTICAL 
ANALYSES SUCH AS MULTIPLE REGRESSIONS, ( BOTH STEP WISE AND 
OTHERWISE ) DISCRIMINANAT ANLAYSES, FACTOR ANALYSIS AND 
OTHER MULTIPLE ANAYSES OF CO-VARIANCE. 

OTHER PROFESSORS DO LITERATURE REVIEW OF THE LITERATURE 
OR THOUGHT PIECES. ARE EITHER OF THESE FORMS OF " RESEARCH 
"BETTER THAN ANY OTHER FORM ? 

CERTAIN PROFESSORS PUBLISH REAM AND REAMS OF PAPER AND 
DATA THAT ARE SIMPLY IRELEVANT AND TEND TO " STRETCH " THE 
DATA FOR ALL IT IS WORTH AND PUBLISH TANGENTIAL, IRRELVANT 
INFORMATION THAT IS NOT DIRECTLY SALIENT TO THE RESEARCH 
CITED. 
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SUMMARY AND CONCLUSIONS 

THIS PAPER HAS ATTEMPTED TO ADDRESS SOME CRUCIAL, 
CRITICAL ISSUES REGARDING THE EVALUATION OF HIGHER EDUCATION 
FACULTY. MUCH WORK NEEDS TO BE DONE AND MANY ISSUES NEED TO 
BE ADDRESSED. 
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